Adaptation Step-by-Step: Challenges for Real-time Spatial Personalization

In this paper we outline challenges for user modeling and personalization with spatial information. To illustrate those challenges we use a use case with a real-time routing system that implements a mobile museum guide for providing personalized tours tailored to the user position inside the museum and her interests. In this scenario we combine on the one hand (1) interactive discovery of user’s interests applied for semantic recommendations of artworks and art-related topics, and on the other hand (2) dynamic step-by-step adaptation of a user’s route through a museum based on her current position and changing interests. For this, the existing CHIP mobile museum guide was extended with a routing mechanism based on the SWI-Prolog Space package. 1 Challenges in Dynamic Spatial Adaptation When a visitor moves around in a museum, exploring the collection of artworks based on her interest in these artworks, many spatial aspects can be considered in the process of recommending the visitor what to see. The same holds for a mobile museum guide, where we exploit techniques of dynamic personalization for such recommendations. Adding the spatial aspects allows to improve the dynamic adaptation further. For example, the assessment of the user’s constantly evolving interests can be improved by step-by-step spatial information. If we include spatial considerations into the many different elements of dynamic adaptation, we have many opportunities, but we also have as many research challenges. The opportunities we illustrate in this paper are based on our experience with the concrete museum demonstrators from our work in the CHIP project and we have elicited research challenges from that same experience. – Dealing with real-time information: When we deal with real-time information in the process of dynamic adaptation, we can consider the user’s position, her context, and her social interaction: • User location: Detecting a user’s location inside the physical space is a first challenging step. In a museum this requires a positioning system that considers the boundaries and constraints (i.e. the walls, doors, stairs) of the space. In our case, we have an indoor space, and therefore methods 2 Willem Robert van Hage et al. using different hardware solutions have been proposed to increase the accuracy of the indoor user positioning. • User context: For the adaptation in the collection-based semantic recommendation process, identifying the relevant context is important. This context typically includes the identification of artworks in her neighborhood, the artworks that have been already seen, the time she has already spent in the museum and additional temporal constraints (e.g. how much time is available), her general interests in art, and potentially also her physiological and emotional state [2]. The main challenge that we met in our case is to know the relevant context from a spatial perspective. • Social interactions: Social interactions can play a role in the adaptation process, e.g. the interactions between people that visit the museum together, and the spatial aspect can impact the process even more. Think of routes that the people take and potential meetings that they might have inside the museum. The social interactions are relevant both as input and output of the adaptation process and including the spatial aspect in these social interactions offers a challenging improvement. – Coping with the limited resources of a mobile device: The dynamic adaptation and context-awareness ask a lot from the infrastructure and the limited resources that mobile devices offer. We have experimented in CHIP with different re-routing algorithms for the purpose of adaptation. The current algorithm can provide re-routing of a tour of artworks based on the user’s position. It would be interesting however to consider more complex algorithms that would also take user preferences into account and possibly decide to add additional artworks to the tour that might be interesting for the user based on the user’s close proximity. – Overcoming mobile platform dependance: The personalization and recommendation process is based on knowledge about the content and the user. To reduce complexity and to ensure reusability of the knowledge representation and inference mechanisms, a flexible web-based approach is required that allows different types of systems to exchange and augment information on users and particular situations [2]. Also, the web-based architecture allows the use of multiple types of mobile devices. – Maintaining distributed user model: The personalization in an application like the ones we consider here, is not a single standalone one. The user model that is relevant for this kind of dynamic adaptation in CHIP asks for the capability to exchange and integrate user model knowledge in a distributed fashion. For this reason, we have chosen a distributed and open web-based solution for user model knowledge representation. – Integration with third-party applications: It would be interesting to consider technologies like Google Goggles to show information about an artwork when the user points with her device to it. This is an example of an integration with third-party applications, and in our case we have chosen an approach that facilitates this space-oriented integration, that as we will show later is mainly based on offering standard interfaces and interoperability. 3 http://www.google.com/mobile/goggles/ Adaptation Step-by-Step: Challenges for Real-time Spatial Personalization 3 – Evaluating in real-life settings: In order to improve this real-time adaptation process, patterns of user’s navigation and evolution of interests would be very helpful. However, collecting large volumes of such data over long periods of time is very difficult. In the remainder of this paper, we describe how we brought the CHIP demonstrators to a next version that includes the spatial dimension. 2 Spatial Personalization in Museums: Related Work Museum curators typically would offer tours on different topics based on the highlights of the collections. Thus, the resulting tours are characterized by a predefined selection and fixed sequence of artworks. An audio tour would still offer a predefined selection of artworks, however it allows for determining your own sequence of artworks. A number of museums, e.g. Tate Modern, Science Museum Boston, are exploring the potential of personalized museum guides. Personalized virtual tours, on the other hand, help visitors construct their own narratives, however they are only limited to online collections. Multimedia guides provide a promising alternative to bridge the gap between the visitor’s interests and the static museum tours by using personalization techniques [5]. An adaptive mobile museum guide acts as a museum expert and provides the user with information adapted to the current situation [2]. For example, the MIT Media Lab audio and visual narration adapts to the user’s interest acquired from the physical path in the museum and length of the user stops. The mobile museum guides developed within Hippie [3] and PEACH [4] projects provide content adaptation based on technical restrictions of specific presentation devices as well as visitor’s preferences and knowledge. The mobile museum guide built within Sotto Voce [1] project takes into account the special needs of groups visiting a museum and facilitates social interaction between group members. Another example for fostering social interaction between visitors is given by the AgentSalon [6] system, and ARCHIE [10] provides a socially-aware handheld guide that stimulates interaction between group members. The Kubadji mobile tour guide uses a collaborative filtering approach for predicting visitor’s viewing times of unseen exhibits from his viewing times at visited exhibits. The context-aware museum guide in [11] is adapting by dropping artworks if the visitor falls behind the tour or is suggesting additional artworks or taking a break at a nearby restaurant if the visitor has extra time. The environment also supports p2p interactions between visitors, to find each other, share ratings and comments about exhibits. Important here is the fact that spatial information is used in relatively limited aspects for adaptation. Usually, the real implementation of such approaches depends on the availability of an indoor localization of people and objects. 4 Virtual Museum (of Canada), http://www.museevirtuel-virtualmuseum.ca/ 5 http://www.media.mit.edu/ 6 http://www.kubadji.org/ 4 Willem Robert van Hage et al. 3 Use Case: Space-CHIP Step-by-Step Route Adaptation The CHIP project is a cross-disciplinary project, combining aspects from cultural heritage and information technologies to provide a personalized access to the museum collection both online and inside the museum, e.g., generating personalized museum tours, getting recommendations about interesting artworks to see, and quickly finding ways in the museum. One important aspect of the project is the use of a common distributed user model, which collects user interaction data and interprets it in terms of user’s interests used further for generating recommendations and personalized tours. Additionally, we also use the e-Culture Semantic Search open API to allow to find semantically related topics and artworks to include in the personalized tours. The Mobile Museum Guide allows users to access their tours created with the online Tour Wizard on their mobile devices in the museum. Details about the design and implementation of the CHIP Art Recommender, Tour Wizard and Mobile Museum Guide (ver 0.1 and ver 1.0) can be found in [9]. The CHIP Mobile Museum Guide (ver 0.1 and ver 1.0) can adapt to the user in many different ways, but mainly based on the known user preferences and availability in the museum. To relate the step-by-step adaptation also to the real physical space, spatial constraints have to be taken into account in the generation of both recommendations and museum tours. Suppose the user follows a tour of recommended artworks. If she provides a rating to an artwork that she sees, the CHIP demonstrator updates the user model and as consequence the list of recommended artworks. However, those versions of the Mobile Museum Guide do not take spatial constraints as well as information about already seen artworks into account for the adaptation. Further we show how we implement the Space-CHIP Mobile Museum Guide that includes adaptation with spatial constrains, http://www.chip-project.org/spacechip. The implementation is based on the SWI-Prolog Space package. The basic approach in the Art Recommender is to recommend based on the estimated likelihood that the user will like the artworks. Even with a theme-based layout of the rooms in the Rijksmuseum, e.g., rooms for the Dutch republic or works by Rembrandt and his pupils, a set of recommended artworks can in reality be distributed over the entire museum. To improve the user experience, we therefore reorder the results of the Art Recommender to allow for an efficient walk from one artwork to the other. Such a route minimizes the walking effort, while maximizing the number of top recommendations. Also, it takes into account optional caps to the walking distance and the number of artworks. This helps the user to decide where to go in limited time. Computing an efficient route through a museum is very similar to the traveling salesman problem. However, it is a significantly easier problem than the general traveling salesman problem. First, if you consider the artworks, rooms, doors, hallways, and stairs to be nodes in a connectivity graph (e.g. Fig. 1), 7 See http://www.chip-project.org/demo 8 http://e-culture.multimedian.nl/ Adaptation Step-by-Step: Challenges for Real-time Spatial Personalization 5 then this graph is not fully connected, as there are walls and floors in the way. Second, from the way in which exhibits are created, it makes sense to view all works from a single room together. Third, floor transitions take a lot of effort. For these reasons there are only a few sensible paths through the museum.